Information selection and fusion in vision systems

Handling the enormous amounts of data produced by data-intensive imaging systems, such as multi-camera surveillance systems and microscopes, is technically challenging. While image and video compression help to manage the data volumes, they do not address the basic problem of information overflow. In this PhD we tackle the problem in a more drastic way. We select information of interest to a specific vision task, and discard the rest. We also combine data from different sources into a single output product, which presents the information of interest to end users in a suitable, summarized format. We treat two types of vision systems. The first type is conventional light microscopes. During this PhD, we have exploited for the first time the potential of the curvelet transform for image fusion for depth-of-field extension, allowing us to combine the advantages of multi-resolution image analysis for image fusion with increased directional sensitivity. As a result, the proposed technique clearly outperforms state-of-the-art methods, both on real microscopy data and on artificially generated images. The second type is camera networks with overlapping fields of view. To enable joint processing in such networks, inter-camera communication is essential. Because of infrastructure costs, power consumption for wireless transmission, etc., transmitting high-bandwidth video streams between cameras should be avoided. Fortunately, recently designed 'smart cameras', which have on-board processing and communication hardware, allow distributing the required image processing over the cameras. This permits compactly representing useful information from each camera. We focus on representing information for people localization and observation, which are important tools for statistical analysis of room usage, quick localization of people in case of building fires, etc. To further save bandwidth, we select which cameras should be involved in a vision task and transmit observations only from the selected cameras. We provide an information-theoretically founded framework for general purpose camera selection based on the Dempster-Shafer theory of evidence. Applied to tracking, it allows tracking people using a dynamic selection of as little as three cameras with the same accuracy as when using up to ten cameras

Dempster-Shafer based multi-view occupancy maps

Presented is a method for calculating occupancy maps with a set of calibrated and synchronised cameras. In particular, Dempster-Shafer based fusion of the ground occupancies computed from each view is proposed. The method yields very accurate occupancy detection results and in terms of concentration of the occupancy evidence around ground truth person positions it outperforms the state-of-the- art probabilistic occupancy map method and fusion by summing

Activity monitoring of people in buildings using distributed smart cameras

Systems for monitoring the activity of people inside buildings (e.g., how many people are there, where are they, what are they doing, etc.) have numerous (potential) applications including domotics (control of lighting, heating, etc.), elderly-care (gathering statistics on the daily live) and video teleconferencing. We will discuss the key challenges and present the preliminary results of our ongoing research on the use of distributed smart cameras for activity monitoring of people in buildings. The emphasis of our research is on: - the use of smart cameras (embedded devices): video is processed locally (distributed algorithms), and only meta-data is send over the network (minimal data exchange) - camera collaboration: cameras with overlapping views work together in a network in order to increase the overall system performance - robustness: system should work in real conditions (e.g., robust to lighting changes) Our research setup consists of cameras connected to PCs (to simulate smart cameras), each connected to one central PC. The system builds in real-time an occupancy map of a room (indicating the positions of the people in the room) by fusing the information from the different cameras in a Dempster-Shafer framework

Image fusion using blur estimation

In this paper, a new wavelet based image fusion method is proposed. In this method, the blur levels of the edge points are estimated for every slice in the stack of images. Then from corresponding edge points in different slices, the sharpest one is brought to the final image and others are eliminated. The intensities of non-edge pixels are assigned by the slice of its nearest neighbor edge. Results are promising and outperform other methods in most cases of the tested methods

Camera selection for tracking in distributed smart camera networks

Tracking persons with multiple cameras with overlapping fields of view instead of with one camera leads to more robust decisions. However, operating multiple cameras instead of one requires more processing power and communication bandwidth, which are limited resources in practical networks. When the fields of view of different cameras overlap, not all cameras are equally needed for localizing a tracking target. When only a selected set of cameras do processing and transmit data to track the target, a substantial saving of resources is achieved. The recent introduction of smart cameras with on-board image processing and communication hardware makes such a distributed implementation of tracking feasible. We present a novel framework for selecting cameras to track people in a distributed smart camera network that is based on generalized information-theory. By quantifying the contribution of one or more cameras to the tracking task, the limited network resources can be allocated appropriately, such that the best possible tracking performance is achieved. With the proposed method, we dynamically assign a subset of all available cameras to each target and track it in difficult circumstances of occlusions and limited fields of view with the same accuracy as when using all cameras

PhD forum: Dempster-Shafer based camera contribution evaluation for task assignment in vision networks

In a network of cameras, it is important that the right subset of cameras takes care of the right task. In this work, we describe a general framework to evaluate the contribution of subsets of cameras to a task. Each task is the observation of an event of interest and consists of assessing the validity of a set of hypotheses. All cameras gather evidence for those hypotheses. The evidence from different cameras is fused by using the Dempster-Shafer theory of evidence. After combining the evidence for a set of cameras, the remaining uncertainty about a set of hypotheses, allows us to identify how well a certain camera subset is suited for a certain task. Taking into account these subset contribution values, we can determine in an efficient way the set of subset-task assignments that yields the best overall task performance

Efficient approximate foreground detection for low-resource devices

A broad range of very powerful foreground detection methods exist because this is an essential step in many computer vision algorithms. However, because of memory and computational constraints, simple static background subtraction is very often the technique that is used in practice on a platform with limited resources such as a smart camera. In this paper we propose to apply more powerful techniques on a reduced scan line version of the captured image to construct an approximation of the actual foreground without overburdening the smart camera. We show that the performance of static background subtraction quickly drops outside of a controlled laboratory environment, and that this is not the case for the proposed method because of its ability to update its background model. Furthermore we provide a comparison with foreground detection on a subsampled version of the captured image. We show that with the proposed foreground approximation higher true positive rates can be achieved

Sub-optimal camera selection in practical vision networks through shape approximation

Within a camera network, the contribution of a camera to the observations of a scene depends on its viewpoint and on the scene configuration. This is a dynamic property, as the scene content is subject to change over time. An automatic selection of a subset of cameras that significantly contributes to the desired observation of a scene can be of great value for the reduction of the amount of transmitted and stored image data. We propose a, greedy algorithm for camera selection in practical vision networks where the selection decision has to be taken in real time. The selection criterion is based on the information from each camera sensor's observations of persons in a scene, and only low data rate information is required to be sent over wireless channels since the image frames are first locally processed by each sensor node before transmission. Experimental results show that the performance of the proposed greedy algorithm is close to the performance of the optimal selection algorithm. In addition, we propose communication protocols for such camera. networks, and through experiments, we show the proposed protocols improve latency and observation frequency Without deteriorating the performance